The Anjials of Statistics 

2011, Vol. 39, No. 5, 2502-2532 

DOI: 10.1214/11-AOS908 

@ Institute of Mathematical Statistics, 2011 



ASYMPTOTIC NORMALITY AND VALID INFERENCE FOR 
GAUSSIAN VARIATIONAL APPROXIMATION^ 

By Peter Hall, Tung Pham, M. P. Wand and S. S. J. Wang 

University of Melbourne, University of Wollongong, University of 
Technology, Sydney, and University of Wollongong 

We derive the precise asymptotic distributional behavior of Gaus- 
sian variational approximate estimators of the parameters in a single- 
predictor Poisson mixed model. These results are the deepest yet 
obtained concerning the statistical properties of a variational ap- 
proximation method. Moreover, they give rise to asymptotically valid 
statistical inference. A simulation study demonstrates that Gaussian 
variational approximate confidence intervals possess good to excel- 
lent coverage properties, and have a similar precision to their exact 
likelihood counterparts. 

1. Introduction. Variational approximation methods are enjoying an in- 
creasing amount of development and use in statistical problems. This raises 
questions regarding their statistical properties, such as consistency of point 
estimators and validity of statistical inference. We make significant inroads 
into answering such questions via thorough theoretical treatment of one of 
the simplest nontrivial settings for which variational approximation is bene- 
ficial: the Poisson mixed model with a single predictor variable and random 
intercept. We call this the simple Poisson mixed model. 

The model treated here is also treated in [7], but there attention is con- 
fined to bounds and rates of convergence. We improve upon their results by 
obtaining the asymptotic distributions of the estimators. The results reveal 
that the estimators are asymptotically normal, have negligible bias and that 
their variances decay at least as fast as m~^, where m is the number of 
groups. For the slope parameter, the faster {mn)~^ rate is obtained, where 
n is the number of repeated measures. 
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An important practical ramification of our theory is asymptotically valid 
statistical inference for the model parameters. In particular, a form of stu- 
dentization leads to theoretically justifiable confidence intervals for all model 
parameters. Unlike those based on the exact likelihood, all Gaussian vari- 
ational approximate point estimates and confidence intervals can be com- 
puted without the need for numerical integration. Simulation results reveal 
that the confidence intervals have good to excellent coverage and have about 
the same length as exact likelihood-based intervals. 

Variational approximation methodology is now a major research area 
within computer science; see, for example. Chapter 10 of [3]. It is begin- 
ning to have a presence in statistics as well (e.g., [10, 14]). A summary 
of the topic from a statistical perspective is given in [13]. Late 2008 saw 
the first beta release of a software library, lnfer.NET [12], for facilitation of 
variational approximate inference. A high proportion of variational approx- 
imation methodology is framed within Bayesian hierarchical structures and 
offers itself as a faster alternative to Markov chain Monte Carlo methods. 
The chief driving force is applications where speed is at a premium and 
some accuracy can be sacrificed. Examples of such applications are cluster 
analysis of gene-expression data [17], fitting spatial models to neuroimage 
data [6], image segmentation [4] and genome-wide association analysis [8]. 
Other recent developments in approximate Bayesian inference include ap- 
proximate Bayesian computing (e.g., [2]), expectation propagation (e.g., [11]), 
integrated nested Laplace approximation (e.g., [16]) and sequential Monte 
Carlo (e.g., [5]). 

As explained in [3] and [13], there are many types of variational approx- 
imations. The most popular is variational Bayes (also known as mean field 
approximation), which relies on product restrictions applied to the joint 
posterior densities of a Bayesian model. The present article is concerned 
with Gaussian variational approximation in frequentist models containing 
random effects. There are numerous models of this general type. One of 
their hallmarks is the difficulty of exact likelihood-based inference for the 
model parameters due to presence of nonanalytic integrals. Generalized lin- 
ear mixed models (e.g.. Chapter 7 of [9]) form a large class of models for han- 
dling within-group correlation when the response variable is non-Gaussian. 
The simple Poisson mixed model lies within this class. From a theoret- 
ical standpoint, the simple Poisson mixed model is attractive because it 
possesses the computational challenges that motivate Gaussian variational 
approximation — exact likelihood-based inference requires quadrature — but 
its simplicity makes it amenable to deep theoretical treatment. We take ad- 
vantage of this simplicity to derive the asymptotic distribution of the Gaus- 
sian variational approximate estimators, although the derivations are still 
quite intricate and involved. These results represent the deepest statistical 
theory yet obtained for a variational approximation method. 
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Moreover, for the first time, asymptotically valid inference for a varia- 
tional approximation method is manifest. Our theorem reveals that each 
estimator is asymptotically normal, centered on the true parameter value 
and with a Studentizable variance. Replacement of the unknown quantities 
by consistent estimators results in asymptotically valid confidence intervals 
and Wald hypothesis tests. A simulation study shows that Gaussian varia- 
tional approximate confidence intervals possess good to excellent coverage 
properties, especially in the case of the slope parameter. 

Section 2 describes the simple Poisson mixed model and Gaussian vari- 
ational approximation. An asymptotic normality theorem is presented in 
Section 3. In Section 4 we discuss the implications for valid inference and 
perform some numerical evaluations. Section 5 contains the proof of the 
theorem. 

2. Gaussian variational approximation for the simple Poisson mixed model. 

The simple Poisson mixed model that we study here is identical to that 
treated in [7] . Section 2 of that paper provides a detailed description of the 
model and the genesis of Gaussian variational approximation for estimation 
of the model parameters. Here we give just a rudimentary account of the 
model and estimation strategy. 

The simple Poisson mixed model is 

(2.1) Yij\Xij,Ui independent Poisson with mean exp(/3o + f^iXij + Ui), 

(2.2) U^ independent N{0,{a^f). 

The Xij and Ui, for 1 <i <m and 1 < j < n, are totally independent random 
variables, with the Xjj's distributed as X. We observe values of {Xij,Yij), 
1 <i <m, 1 < j <n, while the Ui are unobserved latent variables. See, for 
example. Chapter 7 and Section 14.3 of [9] for further details on this model 
and its use in longitudinal data analysis. In applications it is typically the 
case that n. 

Let (3 = (/3o,/3i) be the vector of fixed effects parameters. The conditional 
log- likelihood of (/3,(t^) is the logarithm of the joint probability mass func- 
tion of the Yij^s, given the Xjj's, as a function of the parameters 



m n 



Y.^yv{l3o + /3iX,,) - log(y,,!)} - - log(27ra2) 



i=i j=i 



(2.3) 
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Maximum likelihood estimation is hindered by the presence of m intractable 
integrals in (2.3). However, the ith of these integrals can be written as 



du 



Ui 



0=1 



2Xi 



where, for 1 < z < m, E^^ denotes expectation with respect to the random 

variable Ui ~ N[^i,\i) with Aj > 0. Jensen's inequality then produces the 
lower bound 



exp<^ Y^iYijUi - e^o+^i^'^+^>) + 



2cr2 ' 2A,- 



' lj=l 



2^2 ^ 2Ai 



which is tractable. Standard manipulations then lead to 

(2.4) £(/3,a2)>^(/3,a^/x,A) 

for all vectors /x = (;Ui, . . . , ^rn) and A = (Ai, . . . , A^), where 



(2.5) 



i=i j=i 

^ m 

-Yiog(a^) + y-^E(/^? + ^^) 



i=l 



^ m 



is a Gaussian variational approximation to £(/3, cr^). The vectors fi and A are 
variational parameters and should be chosen to make i{P,a'^ , fj,, X) as close 
as possible to £{(3, cj^). In view of (2.4) the Gaussian variational approximate 
maximum likelihood estimators are naturally defined to be 

(/3,£^) = (/S,^^) component of arg max £(/3, cr^, /x, A). 
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3. Asymptotic normality results. Consider random variables {Xij , Yij , Ui) 
satisfying (2.1) and (2.2). Put 

n n 

Yi, = ^ Yij and = ^ exp(/3o + l^iXij), 
i=i j=i 

and consider the following decompositions of the exact log-likelihood and its 
Gaussian variational approximation: 

i{f3, a^) = ioiP, cT^) + h{(3, T^) + DATA, 

£(/3, a^, A) = £o(/3, a') + ^2(/3, (T^, /I, A) + DATA, 



where 



m n ^ 

4(/3,a2) = ^^yi,(/3o + /3iA,j) - -mlogtr^. 



(3.1) 



i=i j=i 



(3.2) 



4(/3, ^2) = logj exp (Vi.u - 5,e" - l^'^u^^ duj, 
f2(/3,cr^,/i. A) = ^j/iiy,, - Sjexp^/Xj + ^Ajj I 

- + ^0 + 9 



2 

1=1 i=l 



and DATA denotes a quantity depending on the Yij alone, and not on (3 or 
0-^. Note that 

i{P,a'^) = max^(/3,cr^,/i. A) = £o(/9, o"^) + niax£2(/3, cr^, A*, A). 

Our upcoming theorem relies on the following assumptions: 

(Al) the moment generating function of X , (j){t) = i?{exp(tA)}, is well 
defined on the whole real line; 

(A2) the mapping that takes /3 to cp' / 4){f3) is invertible; 

(A3) in some neighborhood of /3° (the true value of /3i), {(P/d/3^) log0(/3) 
does not vanish; 

(A4) m = m{n) diverges to infinity with n, such that n/m — )■ as n — )■ oo ; 
(A5) for a constant C > 0, m = 0{n^) as m and n diverge. 

Define 

l^^. ,2_ exp{-(a^)°/2-/3oX/3°) 

^ ' ct>"{Pi)m)-<t>'m ■ 

The precise asymptotic behavior of /3o, /3i and is conveyed by: 
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Theorem 3.1. Assume that conditions (Al)-(A5) hold. Then 

(3.4) lo-Po= m-'/^No + Op{n-^ + m-^/^), 
where the random variable Nq is normal N{0,{a'^)^); 

(3.5) ^1 - /?? = {mny^/^Ni + Opin'^ + (mn)'^/^}, 
where the random variable Ni is normal A^(0,r^); and 

(3.6) - {a^f = m~^/^N2 + Op{n-^ + m-^/^), 
where the random variable is normal A^(0, 2{(cj^)'^}^). 

Remark. All three Gaussian variational approximate estimators have 
asymptotically normal distributions with asymptotically negligible bias. The 
estimators /3o and have variances of size m~^, as m and n diverge in such a 

manner that n/m — )■ 0. The estimator /3i has variance of size (mn)"-*^. Hence, 

the estimator /3i is distinctly more accurate than either /3o or , since it 
converges to the respective true parameter value at a strictly faster rate. For 
the estimator increasing both m and n reduces variance. However, in the 

cases of the estimators /3o or a^, only an increase in m reduces variance. 

4. Asymptotically valid inference. Theorem 3.1 reveals that /3o, /3i and 
are each asymptotically normal with means corresponding to the true 
parameter values. The variances depend on known functions of the param- 
eters and (p'iPi) and Since the latter three quantities can be 
estimated unbiasedly via 

i=l j=l 

i=i j=i 

and 

^ m n 



mn 

1=1 j=i 



we can consistently estimate the asymptotic variances for inferential proce- 
dures such as confidence intervals and Wald hypothesis tests. For example, 
the quantity appearing in the expression for the asymptotic variance of 
/3i can be consistently estimated by 
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Approximate 100(1 — a)% confidence intervals for /3q, /S^ and {cr'^)^ are 



where $ denotes the A'^(0, 1) distribution function. These confidence intervals 
are asymptotically valid since they involve studentization based on consistent 
estimators of all unknown quantities. 

We ran a simulation study to evaluate the coverage properties of the Gaus- 
sian variational approximate confidence intervals (4.1). The true parameter 
vector (/3q,/35^, (o"^)°) was allowed to vary over 



and the distribution of the Xij was taken to be either A^(0, 1) or Uniform(— 1, 1), 
the uniform distribution over the interval (—1, 1). The number groups m var- 
ied over 100, 200, . . . , 1,000 with n fixed at m/10 throughout the study. For 
each of the ten possible combinations of true parameter vector and Xij dis- 
tribution, and sample size pairs, we generated 1,000 samples and computed 
95% confidence intervals based on (4.1). 

Figure 1 shows the actual coverage percentages for the nominally 95% 
confidence intervals. In the case of the actual and nominal percentages 
are seen to have very good agreement, even for (?7i,n) = (100,10). This is 
also the case for /3q for the first four true parameter vectors. For the fifth 
one, which has a relatively low amount of within-subject correlation, the 
asymptotics take a bit longer to become apparent, and we see that m > 400 
is required to get the actual coverage above 90%, that is, within 5% of the 
nominal level. For {cr'^)^, a similar comment applies, but with m > 800. The 
superior coverage of the Pi confidence intervals is in keeping with the faster 
convergence rate apparent from Theorem 3.1. 

Lastly, we ran a smaller simulation study to check whether or not the 
lengths of the Gaussian variational approximate confidence intervals are 
compromised in achieving the good coverage apparent in Figure 1. For each 
of the same settings used to produce that figure we generated 100 samples 
and computed the exact likelihood-based confidence intervals using adap- 
tive Gauss-Hermite quadrature (via the R language [15] package lme4 [1]). 



(4.1) 




and 
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Fig. 1. Actual coverage percentage of nominally 95% Gaussian variational approximate confidence intervals for the parameters in the 
simple Poisson mixed model. The nominal percentage is shown as a thick grey horizontal line. The percentages are based on 1,000 
replications. The values of m are 100, 200, . . . , 1,000. The value of n is fixed at n = m/10. 
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Table 1 

Definitions of the 0(fc) notation used in the proofs 



Notation Meaning 



0(i) Oj,{m-^^^ +n-^) 

0(2) Op(m-i+n-2) 

0(3) 0{n^^'-^^^^), uniformly in 1 < i < m, for each e>0 

0(4) 0{n^~^), uniformly in 1 < i < m, for each £ > 

0(5) 0(n''~'^''^'), uniformly in l<i<m, for each e > 

0(6) Op{m^^ + n^^^^^^^), uniformly in 1 < i < m, for each e > 

0(7) Op{{m^^ + n"^)n^"*^''^'}, uniformly in 1 < i < m, for each e > 

}, uniformly in 1 < i < m, for each e > 

0(9) Op{{mny^^^ + n^"'^/^'}, uniformly in 1 < i < m, for each e > 

0(10) Op{{m~'^^'^ + n"^''^)?!'^}, uniformly in 1 < i < m, for each e > 

0(ii) Op{(m~^''^n~^ + n~^)n^}, uniformly in 1 < i < m, for each £ > 



In almost every case, the Gaussian variational approximate confidence in- 
tervals were slightly shorter than their exact counterparts. This reassuring 
result indicates that the good coverage performance is not accompanied by 
a decrease in precision. 

5. Proof of Theorem 3.1. The proof Theorem 3.1 requires some addi- 
tional notation, as well as several stages of asymptotic approximation. This 
section provides full details, beginning with definitions of the necessary no- 
tation. 

5.1. Notation. Recall that /3q, 13^ and {(j"^)^ denote the true values of pa- 
rameters and that /3o, /3i and denote their respective Gaussian variational 
approximate estimators. 

The proofs use ^^Of^i^-^''^ notation, for /c = 1, . . . , 11, as defined in Table 1. 

5.2. Formulae for estimators. First we give, in (5.1)-(5.5) below, the 
results of equating to zero the derivatives of £o(/3,o"^) + ^2(/3, cr^. A, ^) with 
respect to /3o, o'^, Aj and /Xj, respectively: 




^ lib 

(5.3) -Y,{l^+^^h=^^ 

i=\ 
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(5.4) - Bi eMRi + ^Ai) - (^^)~^ = 0, l<i<m, 

(5.5) Yi, - Bi expCfii + ^Aj) - {d_^)~^'pi = 0, l<i<m. 

These are the analogs of the hkehhood equations in the conventional ap- 
proach to inference. 

The next step is to put (5.1), (5.2) and (5.5) into more accessible form, 
in (5.6), (5.11) and (5.12), respectively. Adding (5.5) over 1 < i < m and 
subtracting the result from (5.1) we deduce that 

m 

(5.6) E^^ = 0- 

i=l 

Defining 

m n 



^ fit, IL 

i=i j=i 

we deduce that (5.2) is equivalent to 

^ m n 

A + exp(/30) V V exp(C/, + p^X.^) 

mn ^-^ ^-^ 

i=l 7=1 

(5.7) 

1 ™ " / I- \ 

- exp(/3o) ^ ^ Xij exp {%. + -\i + PiXij j = 0. 

i=l j=l ^ ^ 

Define ^j, r/j and Q by, respectively, 
1 " 

(5.8) - 5^ Xij exp(/3?X,,) = (^'(/3?) exp(e,), 
1 " 

(5.9) - ^ Xij exp(^iXij) = (/>'(^i) exp(r/i), 

1- \ 1 " 

expf ^0 + -Ai j - ^{exp(^iXy) - (P{/3i)} 
V y ^ j=i 

(/.(/3?){l-exp(0)} 
1 " 

+ - Ei^^J exp(-/3o° - U^) - </>(/??)} 

(£^n)"^/ij. 



:exp(/3° + [/.) 



(5.10) 
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With probability converging to 1 as n — t- oo the definitions at (5.8)-(5.10) 
are vahd simultaneously for all 1 <i <m, because the variables ^j, r]i and 
Q so defined converge to zero, uniformly in 1 < i < m, in probability. See 
(5.30), (5.31) and (5.25) below for approximations to ^i, rji and d; indeed, 
those formulae quickly imply that each of , 7]i and d equals 0(3) . 

Without loss of generality, (/>'(t) is bounded away from zero in a neigh- 
borhood of /S^. Indeed, if the latter property does not hold, simply add a 
constant to the random variable X to ensure that 4>'{j3i) ^ 0. We assume 
that 13^ is in the just- mentioned neighborhood, and we consider only real- 
izations for which f3i is also in the neighborhood. (The latter property holds 
true with probability converging to 1 as n — t- oo.) The definition of Q at 
(5.10) can be justified using the fact that fli < Yi,, as shown in Theorem 2 
of [7]. 

In this notation we can write (5.7) as 



^ m 

A + 4>'Wi)- V exp(/30 + Ui+ 6) 



i=l 



(5.11) 

- 1 1-- 

= 0'(/3i)— V'exp (3o + Jli + -Xi + T], 

— m ^ — ^ \ — — / 



m 

i=l 



and write (5.5) as 

(5.12) exp(2o + = ewWo +Ui + dW^)- 

Substituting (5.12) into (5.11) we obtain 

^ m 

Aexp(-/30)</.(/5?)-i + ,^'(/5?)</.(/3?)-i-^exp(C/,+eO 

5.13 

^ m 

= ct>'{diWir'-y2exp{Ui + r,i + d)- 
— — m ^-^ 

i=l 

5.3. Approximate formulae for Ui and Aj. The formulae are given at 
(5.16) and (5.18), respectively. To derive them, note that (5.5) implies that 

(l + 0(3))m°)exp(/3o° + C/i) 

- (1 + 0(3) )</>(/??) exp(/30 + ^li) - {nd^r% = 0. 
Here we have used the fact that, by [7], 

(5.14) ^0-/30 = 0(1), |i-/30 = O(i), 

and that by (1.3), maxi<j<m \Xi\ = Op{n^) for all e > 0. Therefore, 

(5.15) (1 + 0(3))exp(C/i) = (1 + 0(3))exp(^i + Qi) + {cnd^)-%, 
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where c = (/>(/3j') exp(/3Q). The result maxi<j<m = Op{(logn)^/^} follows 
from properties of extrema of Gaussian variables and the fact that m = 
0{'nP) for a constant C > 0. Moreover, by Theorem 2 of [7], < A j < . 
Therefore (5.15) implies that maxi<j<„ = Op{(logn)^/^}. [Note that, for 

any constant C > 0, exp{— C(logn)^/^} = n~'^^'°^") which is of larger 
order than for each e > 0.] Hence, by (5.15), 

(1 + 0(3)) exp(C/i) = (1 + 0(3)) exp(^i + ^AO, 

and so, taking logarithms, 

(5.16) C/i = ^, + iAi + 0(3). 
Formula (5.4) and property (5.14) entail 

(5.17) {nXi)-^ - (1 + O(3))0(/3?) exp(^i + + p^) - {nq^Y^ = 0. 

Using (5.16) to substitute Ui + 0(3) for /ij + ^Aj in (5.17) we deduce from 
that result that 

(nA,)-i = (1 + 0(3))</'(/3?) eMU^ + /3o°) + {nd^T^ 

= (l + 0(3))m°)exp(f/i + /?o°), 

where to obtain the second identity we again used the fact that 

max \Ui\=Op{{\ogn)^l'^]. 

l<i<m. 

Therefore, 

A, = (1 + 0(3)){n</.(/3?) exp(f/, + /jg)}"^ 

(5.18) 

= {«</>(/??) exp([/, + /30)}-^ + 0(5) , 

where 0(5) is as defined in Table 1. To obtain the second identity in (5.18) 
we used the fact that maxi<j<mexp(— f/^) = 0(?i^) for all e > 0. 

5.4. Initial approximations to /3o — f^o ctnd /3i — P^. These approxima- 
tions are given at (5.19), (5.21) and (5.29), and lead to central limit theorems 
for f3i — f3i, — (3q and — (ct^)", respectively. To derive the approxima- 
tions, write 7(/3i) = i;^>'(/3i)0(/3i)~^ and note that, defining 0(2) as in Table 
1, we have 

7(|i) = 7(/3?) + (^1 - /3?)7'(/3?) + Op(l|i - /3iT) 

= 7(/30) + {1 + Op(m-i/2 + n-i)}(|i - /3?)7'(/3?). 
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[Here we have used (5.14).] Therefore, by (5.13) and for each e > 0, 

^ m 

Aexp(-/30)</'(;9?)-i +7(/3?)-i;exp(?7, +e.) 

i=l 

= [7(/3?) + {1 + Op(m-i/2 + n-i)}(^i - /3?)y(/3?)] 

^ m 

X —y^exp{Ui + T]i + Ci)- 
1=1 

That is, 

^ m 

01 - /30)y (/3O)- V exp(C/, + 7?, + 0) 
— m ^-^ 

i=l 

^ m 

(5.19) = 7(/3?)- exp(C/,){exp(Ci) - exp(r?i + C.)} 

i=l 

+ Aexp(-/3g).^(/3?)"^+0(2). 
Taking logarithms of both sides of (5.12) we obtain 

(5.20) log{,^(^i)/</)(/3?)} = /3o° -lo + Ui + C^-ni- ^Ai, 
which, on adding over i and dividing by m, imphes that 

- 1 / 1- \ 

log{<^(^i)/</.(/3°)} = /30 - ^0 + - J] ( f/. + C. - 2 

which in turn gives 

1 / 1- \ 

^0 - /3g = -(^1 - /3i°)7(/3?) + - E ( + - - 2^0 + ^(2) 

^ m 

(5.21) = -(^1 - /3?)7(/3?) + - + CO 

i=l 

- |2n<^(/3?)exp(^/30 - ^(a^)") | \o(6), 

where we used (5.18) to substitute for Aj and (5.6) to ehminate /Ij from 
the right-hand side, and employed (5.14) to bound (/3i — Pi)'^. Note too 
that E{exp{—Ui)} = exp{^{a'^)^); a term involving E{ex.p{—Ui)} arises from 
Zili via (5.18). 
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5.5. Approximation to Q. The approximation is given at (5.25). First 
we derive an expansion, at (5.22) below, of /Ij. Reflecting (5.16), define the 

random variable 5i by ^i = Ui — \Xi + 5i. Then, by (5.16), 5i = 0(3). Define 
too ^°fc = E,-^i;exp(/3g + /30Xij) for fe = 0,l,2, and = F,. - S,Vxp(f^i); 
and let Fi denote the sigma-field generated by Ui and Then 
E{/^i I J'i) = and 

Bi = {l+l,- I3l + \{1,- l3lf}Bl 
+ {^1 - /?? + (^0 - /3g)(^i - 

+ i(^l-/3?)2i?,2 + 0(8), 

uniformly in 1 < i < m for each e > 0, where 0(8) is as in Table 1. Therefore, 
Yi, - Biexp{Ui + 6i) 

= Y„ - [{l+lo - + i(|o - (3'o f}Bfo 

+ {A - Pi + do - PlWi - /3?)}i?°i 

X exp(?7:/)(l + 5i + Idf + 0(5)) +nO(8), 
where 0(5) is as in Table 1. Therefore, defining 

X^ = ih - + Plf]B% + % - /3? + (^0 - /3o°)(Ii - /3?)}i?°i 

+ i(|i-/3?)2i?°, 
we see that the left-hand side of (5.5) equals 

Yi, - Bi exp(;7i + 5i)- {g^)~% 

= A,-5°exp(C/,)(<^. + H' + 0(5)) 

- Xiexp(C/i)(l + 5i + \5f + 0(5)) 

-{a^r\U,-\li + 5i)+nOi^^) 
= \- {xieMUi) + {l^rHUi - iAi)} 

-6i{iB^o + Xi)eMUi) + {2.Y'} 

- l6fiBfo + Xi) exp{Ui) + nO(5) + nO(8). 
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Hence, (5.5) implies that 



5^ + 1^2 (i?°o + Xi)exp([/, 



Ai - Xiexp{Ui) - (£2)-i(C/, - A,/2) 



+ 0(., + o 



(5) +'^(8), 



(i?o +X.)exp([/,) + (a2)-i 

which implies that 

_ Ai-Xiexp{Ui) , ^ 
'^-(50o + X.)exp(f/.)+''(^) 

= {nexp(/3O)0(/3O)}~HA.exp(-f/,) - x^} + 0(4) 

= {nexp(/3o°)<^(/3?)}-iA,,exp(-^7,) - (|o - /5o°) " (^i " l^iMf^i) + 0(4)- 
Here we have defined 0(4) is as in Table 1 and have used the fact that 



n 



-ii?o^ = exp(/3o")</>(/3i') + 0(3) and 



n-'Bf, = exp(/30),/.'(/3?) + 0(3) = exp(/30),/.(/3?)7(/3?) + 0^,y 
Therefore, 

= Ui- ^ A.j + (^j 

= ^7, + {nexp(/3O)0(/3?)}-^A,exp(-;7,) 

(5.22) 

- do - /3o°) - ih - f^iWi) + 0(4) 
= Ui-U + {neM^3oWl)}~^\eM-U^) + 0(4), 

where to obtain the second identity we used (5.18) to place into the 
remainder, and to obtain the third identity we used (5.21) to show that /3o — 
f^o + (/3i — /3i)7'(/3i) = U + 0(4). Here we have used the property, deducible 
from (5.10), (5.16) and (5.18), that Ci = 0(3) and C = 0(4). 

The next step is to substitute the right-hand side of (5.22) for juj, and the 

right-hand side of (5.18) for Aj, in (5.10), and derive an expansion, at (5.25) 
below, of Q. We obtain 



_ 1 " 

[1 + {nexp(/3O)0(/3?)}-^A,exp(-C/,) - [7]- ^{exp(^iX,,) - </.(^i)} 

= -m) Ui + la) + - j2iY^J exp(-/3g - u,) - m)} 

V / ^ j=i 

- exp(-/30 - Ui){d^n)~^Ui + 0(5), 
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whence 

m°)(c. + ^cf) 

(5.23) - [{nexp{P^)<^){P^)}-^A,eM-U^) - U] 

1 " ^ 
x-V{exp(/3iXi,)-(/.(/3i)} 
n ^-^ — — 
i=i 

- exp(-/30 - Ui)(g^nY^\Ji + 0(5). 
However, defining 

n 

(5.24) D,^{h) = - Y,{X% exp(6X,,) - <a(^')(&)} = 0(3) 
for /c = 0, 1, 2, and Aj = Yi, — B^^ exp(?7j), we see that 

n n 

Y,{Yi, exp(-/30 - t/.) - </.(/3?)} - ^{exp(|iX,,) - </<(|i)} 
i=i j=i 

n 

= ^{y,,exp(-/30-[/0-m°)} 
i=i 

- n{ Ao(/3?) + (|i - /3?) Ai(/3?)} + 0(3) 
= Aiexp(-/30 - C/.) - n(|i - /5?)Ai(/3?) + 0(3), 
and so, by (5.23), 

= n-iexp(-/30 - f/i)[A,{l - </)(/?0)-i Ao(/??)} - (^')-'f/i] 
- (^1 - /3O) Ai(/3?) + f/Ao(/3?) + 0(5). 

Therefore, 

</.(/3?)C. = n-^exp(-/30 - C/.)[A.{1 - (/>(/3?)-i Ao(/3?)} - (2')"'^/^ 

(5.25) - (^1 - /3?)Ai(/3i°) + f/Ao(/3?) 

- \<\>{^\Y\n-^ exp(-/3o° - C/OAJ^ + 0(5). 
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Result (5.25), and the fact that n/m — ?• as n — ?• oo, imply that 

30 ^1 



m)- E f^^c. = - — ^^^^^ E u! eM-u.) 

i=l 2=1 



2m 

i=l 

(5.26) 

+ Op(n ^) 



Here we have used the fact that E{Uf exp{-Ui)} = exp{Ua'^)^){a'^)^{l + 

5.6. Initial approximation to o"^ — (cr^)^ . Starting from (5.20), using (5.21) 
to substitute for /3o — /3o> using (5.18) to substitute for Aj and defining 
tj = J2i ^iid C = J2i Cii we obtain 

li = U^ + Q- ^li - log{,/.(|i )/(/.(/??)} - (^0 - /3o°) 

= ^7, + - - (^1 - /3?)7(/5?) - (^0 - /3g) + 0(2) 
(5.27) - ~ _ _ 

= U, + C^- {2n</)(/30) eMUi + /3o°)}~' " (f^ + C) 

+ {2n<A(/??) exp(/30 - ^(a^)^)}-^ + 0(6). 
Hence, squaring both sides of (5.27) and adding, 



^ m ^ m 

-E^' = -E(f^^+c^-^-cT 

Ti ^ — ^ — m ^ — * 



m — m 

1=1 i=l 

m 

(5.28) - {mn</.(/3?) exp(/30)}-^ E^^P^-^^)^^^ + 0-^^-0 

1=1 

+ 0(6)- 

Combining (5.3), (5.18), (5.25) and (5.28) we deduce that 
m ^-^ — 

^ m 

(5.29) = {ay + - + Q-U- C? - (^')°} 



i=l 



+ \ n<A(/3?)exp( - ^(<t2)o) | \i + (a'f) + 0^,y 
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5.7. Approximations to and rji. The approximations are given at (5.30) 
and (5.31), respectively, and are derived as follows. Note the definition of 
Dik{b) at (5.24). In that notation, observing that n/m^O and recalling 
(5.14), it can be deduced from (5.8) and (5.9) that, miiformly in 1 <i <m, 

(5.30) = 0'(/3?)-^Ai(/3?) - H'A'(/??)-'Ai(/3?)}' + 0(5), 
r?, = 0'(/3?)-i[Ai(/3?) 

(5.31) + (|i - /??){A2(/3?) - </.'(/??)-V"(/3?)Ai(/3?)}] 

-i{<^'(/30)-iAi(/3?)}^ + O(5). 
Result (5.30) is derived by writing (5.8) as 

(5.32) <A'(/3?)-i Ai(/3i°) = exp(ei) - 1 = + ^Cf + 0^^^), 

and then inverting the expansion. [The result maxi<j<m = Op{l), in fact 
0(3), used in this argument, is readily derived.] To obtain (5.31), note that 
the analog of (5.32) in that case is 

(5.33) <l)'0i)-^Daili) = exp(r?i) -1 = 7?, + Irjf + Opi\i]if), 
and that, uniformly in 1 <i <m, 

= WiPl) + (|i - /3?)0"(/3?) + 0(2)}"' 
X {Ai(/3?) + ill - /3?) A2(/3?) + 0(7)} 

(5.34) = </.'(/3?)-Hl - (^1 - Z??)-/-' (/??)- V"(/3?)} 

X {Ai(/3?) + (^1 - /3?) A2(/3?)} + 0(7) 
= ,^'(/30)-i[Ai(/3?) + (^1 - /3?){A2(/3?) - </''(/??)-V"(/3i°)Ai(/3?)}] 

+ 0(7). 

Result (5.31) follows from (5.33) and (5.34) on inverting the expansion at 
(5.33). 

5.8. Another approximation to /3i — f^^, and final approximations to /3o — 
Pq and — (o-^)*^. Next we use the expansions (5.30), (5.31) and (5.25) of 
i^j, r]i and Q to refine the approximations derived in Section 2.3. The results 
are given in (5.41), (5.42) and (5.46) in the cases of /3o — Pq, Pi — Pi and 
£7^ — {(7^)^, respectively. 

It can be deduced from (5.31) and (5.25) that 

X ™ /I \ 

(5.35) -J^exp(A + ??i + G) = exp -(ct2)0 +0(3). 
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By (5.30), (5.31) and (5.25), 

^ m 

— Vexp(C/i){exp(^i) -exp(r/i + Ci)} 

1 



m ■ 
1=1 



m 

(5.36) 



- Vexp(C/i) 



- r/i - + - iVi + Cif} 



+ 0(5) 



(5) 



1 f 1 1 

= ^ exp(C/,) i'^V^C^ + C?) + O 

1=1 J 

+ Op(||i-/30|n^-(V2)). 
Defining 0(9) as at Table 1 we deduce from (5.25) that 



^ m ^ ^ m 

-^eMU^K^ = --<A(/??)^'^ 5]exp(-2/3g - U,)A^ 

i=l i=l 



(5.37) +Op{(mn)^i/2} + 0(5) 

= -(2n)-V(/3?)-'exp(-/30) + 0(9), 

where we have used the fact that n/m^ and, since 1^,, conditional on J^i, 
has a Poisson distribution with mean i?^Qexp(C/j), then 

E{eM-Ui)A^} = E[exp{-U^){Yi, - E{Yi. \ 

= E{eM-Ui)BleMUi)} 
= E{Bl)=neMfil)ct>{fil). 

Similarly, 

^ m ^ m 

-Y^eMU^)C? = <A(/3?)"'^E«^P(-2/5S - U^)A^ + 0(9) 

1=1 i=l 

(5.38) 

= n-V(/3?)-'exp(-/3g) + 0(9). 
Moreover, since by (5.31) and (5.25), 

rii = ,/.'(/3?)-iAi(/3?) + 0(4), Ci = </'(/3?)"^^"' exp(-/30 - C/,)A, + 0(4), 
and for k>0, 

E{eMUi)D,M) exp(-C/,)AJ = E{DiM)E{A, \ J",)} = 0, 
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then 

^ m 

(5.39) -5^exp(C/,)r?^0 = O(5). 

1=1 

Together, (5.36), (5.37), (5.38) and (5.39) imply that 

^ m 

— Vexp(C/j){exp(^i) - exp(r/i + (i)} 
m ^-^ 

1=1 

= (2n)-V(/3?)-'exp(-/30) - (2n)- V(/S?)"' exp(-/30) 

(5.40) 

+ 0(9)+Op(||i-/5?|n-"(i/2)) 
= 0(9) + Op(|^i-/3?|n^-(^/')). 

Combining (5.19), (5.35) and (5.40), and noting that A = Op{(mn)~^/^} 
and n/m — 7- 0, we deduce that 

(5.41) ^1-/30^0(9). 
Together, (5.21) and (5.41) imply that 

(5.42) Iq- f3^ = U + (- con-i + Op{m-^/^ + n"^), 
where 

co = {2,/.(/3?)exp(/30-|(a2)0)}-\ 
Result (3.4) of Theorem 3.1 is a direct consequence of (5.42) and the property 

m 



m 

i=l 



(5.43) - ^4>iP^^)-^E{n-^ exp(-/?o - Ui)Ai}^ + Op(n-i) 

= Con"^ +Op(n"^). 
Results (5.25) and (5.41), and the property 

E{exp{-2Ui)A^i} = E{B%eM-Ui)} = neMl^o + ki^^fW^i)^ 
imply that 

^ m ^ m 

-Y.^f = <^(/3?)-'^ J]exp(-2/3S - 2[/,)Af + 0^(1) 

i=\ i=l 

(5.44) =„-i<^(/30)-i^^p| 1(^2)0 _^o|^^^(„-i) 
= 2con~^ + Op{n~^). 



m 

i=l 
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By (5.26), 

i E = -'-mr' exp - (i + liar) 

(5.45) 

+ Op{n ). 
Together, (5.43)-(5.45) give 

^ m 

7T) — ^ 

771 777, 

m ^-^ m ^-^ 

i=l i=l 

2 ™ 

+ -E^^Ci-2f^C + 0p("^~') 
1=1 

- f^iU! - ia'f) + 2n-% - 2n-ico(2 + {a'f) 

i=l 

+ Op(m~i/2 _^n-^) 

-E(f^^-(^Y)-2n-%(l + (cTY) 



m 

i=l 



Hence, by (5.29), 

(5.46) £2 - (a^ = i- ^(C/2 - (a^) + Op(m-V2 + ^"1). 

Result (3.6) of Theorem 3.1 is a direct consequence of (5.46). 

5.9. Final approximation to Pi — Pi- Our first step is to sharpen the 
expansion of (5.5) at (5.15); see (5.50), which leads to (5.55), the principal 
analog of (5.15). 

Recall that 

n 

A, = Y„ - exp(/3o° + Ui)Y,^MPiXij) 

(5.47) 

= Y,.-^i,^{Ui)Bl. 
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Also, in view of (5.41) and (5.42), 

n 

Bi = exp(^o) X]^^P(^i^«i) 
i=i 

= exp(^§){l + 00 - Pi) + \{h- Pi? + \{lo- /5o)'} 

j=i ^ J 

( ^ 1 _ 

= exp(/30) ^ 1 + (^0 - + 2 (^0 - /^S)' 

+ ^(Ii - /??)'^^' + 00 - PlMi - /3?)X,,| exp(/3?X,,) 
+ Op(?n-VV + n^-(5/2)) 

= |l + (^0 - /3o°) + ^(lo - P'of + lilo- /3g)'}5°o 

+ {1 + (|o - Plmli - + 101- PifB^2 + 0(10), 

where O(io) is defined in Table 1. Hence, recalling that (5j = /I j + ^Aj — Ui, 
we see that, for each e > 0, we have, uniformly in 1 < i < n, 

Yi, - Biexp{6i + Ui) 

= Yi,-B%eMS^ + U^) 

(5.48) - [{(^0 - Pi) + h0o- PD' + 100- Pl)'}Bl 

+ {1 + (^0 - Pl)}0i - PM + k0i - Pi)'B'^2] 
X exp(5i + f/i) + 0(10). 
Combining (5.47) and (5.48) we obtain 
Yi,-Bieiip{6i + Ui) 

= Ai- exp(C/i)({exp(5,) - 1}B% 

+ [{0o-Pl) + Ulo-Pl)' 
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+ {l + (^o-/3g)}(^i-/3?)S[\ 

+ i(|i-/3?)2i?0]exp(<^,)) 

+ 0(10)- 
Therefore, (5.5) implies that 

{d^r% = A, - eMUi){{eMS^) - ^}B% 

+ [{(^o-/3o°) + i(Io-/3o°)' 

+ {l + (^o-/3o°)}(&-/3?)^°i 

+ i(^i-/3?)2BO]exp(5,)) 

+ 0(10), 

or equivalently, 

exp(i7,)({exp(5,)-l}i?l{, 

+ exp(5i)[{(^o - /5o°) + ^(^0 - /3o°)' + |(^o - f3°of}B^o 

(5.49) 

+ {1 + (^0 - P'o)}0i - Pl)B^i + ^(^1 - P'ifB^2]) 



+ {g_^)-\5i + U,- = A, + 0(10). 

Substituting the far right-hand side of (5.18) for Aj in (5.49) we deduce 
that 

exp(<5,) - 1 + exp(<^,){(|o - /3o°) + \(h - ^1? + 01 - PUB^/BI)} 

(5.50) +{^^BleMUi)}-\5^ + Ui) 

= {i?°oexp(C/,)}-^A, + 0(ii), 

where 0(ii) is as defined in Table 1. Result (5.50) implies that 

(5.51) 5i + \5lGi2 + l6fGi3 = Gi + O^n) , 
where, putting 

G.i = 1 + (^0 - /3o°) + 100- /3o°)' + (^1 - l3'i){Ba/B^,) 

(5.52) _ _ _ 

+ {a^B^o^xp{Ui)}-\ 
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we define Gi, Gi2 and Gjs by GizGn = 1, 

(5.53) G,2Ga = 1 + (|o - + (^i - /3?)(i?.i/i?°o), 

(5.54) 

- m - f^o) + Hlo- + 01 - Pl){Bl/Bl)}. 
Solving (5.51) for 5i we deduce that, for each e > 0, 

(5.55) 6i = Gi — \Gi2G'j — {^Gis — ^G^2)G'f + 

uniformly in 1 < i < n. Now, Gn, Gi2 and Gis each equal 1 + Op{m~^^^ + 
n"-i). Therefore, gGis-^Gfa = -i + Op(m-i/2 +n=-i). Using (5.52), (5.53) 
and (5.54) we deduce that 

Gi2 = l- {i.^B^oexp{Ui)}-^ + Op(m-i + n'-^), = i/,, + 0(n), 

where 

i/i = [{BOoexp(C/,)}-^A, - {a^BfoeMU^)}-^U^ 

(5.56) - {(^0 - /3o°) + ^(lo - + 01 " /3?)(i?°i/i?°o)}] 

X [1 - (|o - /3o°) - (^1 - f^'^mi/Bf,) - {d'B^,eMU.)}-']. 

Note too that Gi2Hf = Hf + Op{m~^/'^n^~^ + n^~^). Combining the results 
from (5.55) down we see that 

(5.57) 6i = Hi-^Hf + lHf + 0^u)- 

Note that, as a — > 0, exp(a — + ^a'^) — 1 = a + O(a^) as a — ^ 0. This 
property, (5.57) and the fact that Hf = Op{n^~'^) imply that 

(5.58) exp{6i)-l = Hi + 0(^u)- 

The formula immediately preceding (5.19) is equivalent to 

^ m 

{1 + Op{m-'/^ + n-i)}7'(/30)(^i - /J?)- V exp([/, + r?, + C^) 

— m ^-^ 

i=l 

(5.59) =Aexp(-/30),/)(/3?)-i 

^ m 

+ ^(^?)- Z^^^'^P^^*) " ^''P^^^ + '^*)} exp(f/i). 

Since r/j and Q both equal 0(3) [see (5.25) and (5.31)], and Yl^i 6xp(?7t) = 
E{exp{Ui)} + Op(l) = exp{ (0-2)72} + Op(l), then (5.59) implies that 

{1 + o,(l)}y(/3?)(^i - /3?)exp{(cT2)V2} 
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(5.60) =Aexp(-/3g)</.(/3°)-i 

^ m 

+ 7(/5?)- V{exp(ei) - exp(r?i + Q)}exp{Ui). 
m ^-^ 

1=1 

Formulae (5.8) and (5.9) are together equivalent to 

(5.61) <A'(/3?){exp(e.) - 1} = iX]{X,,exp(/3?X,,) - <p'if3f)}, 

1 ^ ^ 

(5.62) <P'{Pi){exp{iji) - 1} = - exp(^iX,,) - 

Result (5.62) implies that, for each e > 0, 

{,^'(/3?) + 0,(1^1 -/3?|)}{exp(7?,)-l} 
1 " 

= - exp(/3?X,,) - </>'(/3?)} + Opd^i - /3iV--(i/2)), 

uniformly in 1 <i <n. Therefore, since r]i = 0(3) [see (5.31)], then 

1 " 

0'(/3O){exp(r?,) - 1} = - ^^{^ii exp(/3?X,,) - 0'(/3?)} 

+ Op(||i-/3?|n^-(i/2)), 

which in company with (5.62) implies that 

(/)'(/3?){exp(r?,) - exp(^,)} = Op(|^i - f3f\n^~^'^'^), 

uniformly in 1 <i <n. Hence, since iji = 0^3) and Q = 0(3) [see (5.25) and 
(5.31)], 

exp(Ci) - exp(r?i + Q) = {exp(^i) - exp(r/i)} exp(Ci) 

+ exp(fj){l -exp(Ci)} 

(5.63) 

= exp(^i){l - exp(Ci)} 
+ Op(||i-/3iV^-(i/2)), 
uniformly in i. Combining (5.60) and (5.63) we deduce that 

{l + o,(l)}y(/3?)(|i-/5?)exp|i(a2)o| 
(5.64) =Aexp(-/30)</)(/3?)-i 

^ m 

+ 7(/3?)- V exp(e. + U,){1 - exp(O)}. 

m. ^ — ' 
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Next we return to (5.10), which we write equivalently as 

1 " ^ 

(/.(/3?){1 - exp(O)} = exp(^o - 1^1 + J]{exp(^iX,,) - </>(^i)} 



n 
i=i 



1 " 

(5.65) - - Y^iYij exp(-/30 - U,) - <A(/3?)} 



n . 



+ (£2n)-i^iexp(-/3°-C/i). 

So that we might replace /3i by fS^ on the right-hand side of (5.65), we 
observe that 

^ n 1 " 

- 5^{exp(^iX,,) - <A(|i)} = - J]{exp(/30X,,) - <A(/3?)} 
i=i i=i 

(5.66) 

+ Op(|^i-/3?|n^-(i/2))^ 

Combining (5.64)-(5.66) we obtain 
{l + o,(l)}7'(/3?)(^i-/3?)exp|i(a2)0 



^2 

= Aexp(-/3O)0(/3?)-i 
</'(/3i)^ m ^ 

(5.67) 

exp(^o - /3o° + '^O- j;{exp(/3?X,,) - ./.(z??)} 



1 

-5^{y,,exp(-/30-C/.)-m°)} 

+ (£'n)-i/i,exp(-/30-?7i) 



n 



(RecaU that 7 = (j)'4> ^, and so 'y/(j) = ^0 

Since exp(e») - 1 = Ai(/3i°)0'(/3?)-^ [see (5.8)] and ^0 - /3o° = Op(?«"^/^ + 
re"^) [see (5.42)], then 

-. m 1 

- Vexp(Ci + Ui) exp(^o - /3o° + ^i)- V{exp(/3?X,,) - </<(/30)} 
i=\ j=i 
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1 + (^0 - Pi) + ^(^0 - /3o°)' [- E«^P(^^ + ^^ + f/.)Ao(/5?) 
(5.68) +Op{m~^/'^ +n 



' 1=1 



l + (Io-/3o°) + ^(2o-/3o°)' 



^ m 

X - ^exp(5i + U,){1 + Ai(/5?)</''(/5?)-^}Ao(/5?) 



m 
1=1 



Likewise, 



^ m 1 

- 5^exp(Ci + Ui)- J2{yiJ exp(-/3o° - f/.) - <A(/3?)} 
i=i ^ j=i 

^ m 

(5.69) = - ^exp(C/,){l + Ai(/5?)</''(/5?)-i} 



m 

1=1 



X {n-iA,exp(-/30 - Ui) + Ao(/3?)} 
and, since = [s^e (5.6)], 

— Vexp(e» + ^7i)(?^n)-i/2iexp(-/3|J - Ui) 
m ^-^ — 
1=1 

^ ra 

-2 exp(^i - /3|J)/Ii 

i=l 

^ m 

(5.70) = exp(-/30).^ + Ai(/3?)</''(/5?)^n/^. 

— i=l 

^ m 

= exp(-/30)</.'(/3?)"i.^j;Ai(/3?)/2. 



i=l 



= Op{m-^l^n~^l^). 
Combining (5.67)-(5.70) we see that 

{1 + Op(l)}7'(/3?)(^i - /3?)exp| 1(^2)0 



2 



aO\-l 



Aexp(-/?o"),/.(/3i' 
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^ ,<^'(/3?) 



l + (Io-/3o°) + ^(Io-/3S)' 



my 

^ m 

(5.71) X - Vexp(5, + [/,){! + Ai(/5?)</''(/5?)-^}Ao(/3?) 

rrj ^ — * 



m 

1=1 



^ m 

- exp(-/30)- ^^{1 + Ai(/3?)0'(/3?)"'} 

i=l 

X {n-^Ai + exp(/30 + C/0 Ao(/5?)} 

Using the fact that E{/S.i \ Fi) = and Dii{l3^) = 0(3-) it can be proved 
that, for all e > 0, 

^ m 

— 5]exp(-/3O){l + Ai(/3?)0'(/3?)-nAi 

(5.72) 

^ m 

= exp(-/30)— V A, + Op(m-V2^-i). 

Also, 

A-.A.xp(-ft>(,?)-- -^'J't''^?' ^f:A. 

(5.73) =,^(/30)-iexp(-/30) — 

mn 

X 



Moreover, using (5.42) and the fact that Ao(/3i) = 0(3) and £;{Ao(/5i) 
Ui\ = 0, it can be shown that 

^ m 

0o-fil)-Y.''MUi){l + Da{PlW{Plr^]D,o{fil) 

— m. 



m 
1=1 



(5.74) = Op{(m"i/2 + ^-1) . (rn"i/2n^-(i/2))} 

= Op(m-i/V-i). 
Combining (5.71)-(5.74) we deduce that 

{l + o,(l)}7'(/3?)(^i-/3?)exp|i(cT2)0 
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(5.75) 

x{l + Ai(/3?)0'(/3?)-nAo(/3?) 

Using (5.58) to substitute for exp((5j) — 1 in (5.75), and noting that Dik{j3^) - 
0(3) for k = 0,1, we deduce from (5.75) that 

{l + o,(l)}7'(/3?)(^i-/3?)exp|i(a2)0 

(5.76) 

W) 

where H = {Hi, . . . , Hm), Hi is as defined at (5.56), and, given a sequence 
of random variables K = {Ki, . . . , Km), we put 



^ m 

V^(i^) = - ^ exp(C/Oi^.{l + Ai(/3?)0'(/3?)-^}Ao(/3?). 



m 

i=l 



Note again that |L>jo(/3i)| = 0(3), and the dominant term on the right- 
hand side of formula (5.56) for Hi is {B^q exp(C/i)}~^Aj. Moreover, |/3o — 
= 0p(m-i/2 + n-i) [see (5.42)], |^i - /S?] = Op{(mn)-i/2 + n"-(3/2)} [gge 
(5.41)], 

{£'S?oexp(t/.)}-' = {n(a2)V(/3?)exp(/50 + [7,)}"^ + ©.(n^-^^/^)) 

and 

i?°i?°0-i = <^'(/3°)<A(/3?)-^+O(3). 

Combining these properties we deduce that (5.76) continues to hold if, on 
the right-hand side, ipi^H) is replaced by ip{H') where H' = {H[, . . . , H'^J 

and H'i = h\^^ - i/f ^ - nf^ , with 

) = exp(C/,)}-iA,[l - (^0 - /3o°) - {n{a^fm)^Mm-\ 

H\^^ = {^^B%eMU^)r'Ui 
and 

Hf^ = do - /3g) + i(|o - /3S)' + (^1 - /??){</>' (/3?)/0(/3?)}. 

(Note that H^ does not depend on i.) It can be proved from the properties 

E{Ai \Ti) = and | Ao(^i)| = 0(3) that, with denoting {h[^\. . . , H^l^), 
we have 

(5.77) ij{H^^'>) = Op{m-^/^n-^). 
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More simply, since E{Ui \ Xn, . . . , Xm) = 0, then 

^ m 

V'(i/(2)) = -J](52i?,'l,)-iC/.{l + Ai(/3?)</''(/3?)-nAo(/3?) 

(5.78) 



Oj,(m-V2n-3/2). 



Furthermore, writing 1 = (1, . . . , 1), an n-vector, and noting that the prop- 
erties ^{Afc(/3?) | = 0, var{Afc(/3?) I = 0{n~^) and ^{exp(?7,)} = 
exp(i((T2)'^) imply that 

^ m 

V^(l) = - ^ exp(C/,){l + Ai(/3?)0'(/3?)"'}Ao(/3?) 



-1/2) 



i=l 



n-H'^'(2/3i")<^'(/?i)^ - W)}exp( -(a 



2\0 



we obtain 



iPo - /3o") + ;.(/3o - /3o")^ + (/3i - m^'{P"i)/m)} 



V^(l) 



n 



-[c/.'(2/3O)0'(/3?)"i-<^(/3?)]exp -(a 



(5.79) 



+ Op(m-i/2^-i/2^n-3/2) 



(2o - /3o) + 2 (§0 - /3o°)' + (2i - /3?){<^'(/3?)/'^(/3?)} 



= Op(m~^/2n~^). 

To obtain the last line here we used (3.4) of Theorem 3.1, already proved in 
Section 5.8 above. 

Combining (5.77)-(5.79), and noting that the function ■0 is linear, so that 
iP{H) = V^(/7«) - V'(i?^^^) - V'(^^^^), 

we deduce that 

(5.80) {1 + Op{l)W{Pl)ili - /3?) exp(l(a2)0) = A' + Op{(mn)-V2 ^ ^-2|^ 
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Furthermore, the random variable A', defined at (5.73), is asymptoticahy 
normally distributed with zero mean and variance 



mn \ [ 4>{Pi 
= (mn)-^exp(-2/3|J)S 



{mn)-' exp Qca^)" - /Jg) E [|xn - ^MP'^Xn) 



= (mn)-S'(/3?)2exp{(cT2)0}r2, 

where is as at (3.3). Result (3.5) of the Theorem 3.1 is implied by this 
property and (5.80). 
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